Sasbdb load by wpotrzebowski · Pull Request #3913 · SasView/sasview

wpotrzebowski · 2026-04-01T09:15:12Z

Description

This branch adds Load from SASBDB: users can enter a SASBDB dataset ID, fetch metadata and intensity data from the SASBDB REST API, download a local copy, load it through the existing File data pipeline, and enrich the loaded Data1D objects with SASBDB-derived metadata (sample, instrument, publication, Rg, I(0), Dmax, MW, etc.) where the API provides them.

Implementation highlights

sasbdb_api: HTTP client for https://www.sasbdb.org/rest-api using requests — metadata fetch, URL discovery for intensity files, streamed download to a temp path, robust parsing into SASBDBDatasetInfo (dataclass). Validates 7-character entry codes (e.g. SASDN24) before calling the API; does not guess download URLs when metadata lacks a file link.
SASBDBDownloadDialog: Simple UI (dataset ID, Download, progress, status, Help/Cancel); validates ID up front; on success returns filepath + parsed info for loading.
sasbdb_loader: Loads the downloaded file via filesWidget.readData, then populate_metadata() merges API fields into data.sample, data.run, data.instrument, data.title, and meta_data['SASBDB_*'] keys. Surfaces a warning if readData returns no datasets.
sasbdb_display: Formats metadata for dialog status, logging, and Data Explorer summary panels. Dict-line cleanup applies only to SASBDB datasets so non-SASBDB summaries are unchanged.
GuiManager.actionLoad_SASBDB: Wired from File → Load from SASBDB… (MainWindowUI.ui); thin handler that opens the dialog and delegates to load_downloaded_dataset.
GuiUtils: retrieveData1d / retrieveData2d call append_sasbdb_data_summary() when SASBDB_code is present — SASBDB display logic lives in the SASBDB package, not in GuiUtils.
Docs: User menu/tools Sphinx sources and in-app help sasbdb_download_help.rst.

How Has This Been Tested?

Manual (recommended):
1. File → Load from SASBDB; enter a known valid SASBDB entry ID.
2. Confirm download completes, data appears in Data Explorer, and plots load as for a normal ASCII/dat file.
3. Inspect dataset details / summary text for SASBDB fields (code, Rg, publication, etc.) when returned by the API.
4. Try an invalid ID and confirm a clear error (404 / network).
5. Offline: confirm failure is handled without crashing (message or log).

Review Checklist

Security / network: Timeouts, HTTPS-only usage, and user-visible errors for failed downloads are acceptable.
Metadata mapping: SASBDB → Sample / Source / meta_data keys do not break loaders or perspectives.
SASBDBDownloadDialogUI.ui is accompanied by regenerated Python UI (convertUI.py / project build) if required.
Documentation (menu_bar.rst, tools.rst, sasbdb_download_help.rst) matches the menu label and workflow.
requests remains available in the installed dependency set for end users (not only dev tooling).

Documentation (check at least one)

There is nothing that needs documenting
Documentation changes are in this PR
There is an issue open for the documentation (link?)

Installers

There is a chance this will affect the installers, if so
- Windows installer (GH artifact) has been tested (installed and worked)
- MacOSX installer (GH artifact) has been tested (installed and worked)
- Wheels installer (GH artifact) has been tested (installed and worked)

Licensing (untick if necessary)

The introduced changes comply with SasView license (BSD 3-Clause)

…_load

… sasbdb_load

jamescrake-merani

I've made a few comments on the code I'd like you to take a look at.

I've tested with a couple of IDs. The first one worked, but the second one (SASD2B2) had an error that the identifier was invalid, but I'm not sure it is? Unless I am doing something wrong.

Also, are you intending these changes to go into the next major release? There are some things in here that would need to change with the refactoring project, but if its intended to go into the next release then we probably shouldn't do that work now.

Let me know if anything in the review is unclear. I realise some of the changes I've talked about are not straight forward. If it helps, I'm happy to jump on a meeting to discuss, or try to do some of the changes myself.

jamescrake-merani · 2026-08-12T08:44:52Z

+def metadata_summary(info: SASBDBDatasetInfo) -> str:
+    """Format dataset metadata for dialog status and logging."""
+    lines = []
+    if info.title:


I'm not very comfortable with how long, and repetitive this function is but I also can't suggest any easy way of fixing it.

What I would say though is that it might be better if this is a method within the SASDBDatasetInfo class. You could then use a decorator to give each field a full string name to be printed, and then have a print function within the same class to loop over every field in the dataclass, check if its not none, and print it if so.

jamescrake-merani · 2026-08-12T08:47:39Z

+    return f"{rg_text} Å"
+
+
+def _format_i0_line(i0, i0_error=None, style="label") -> str:


This appears to have the same pattern as the _format_rg_line function. Would it be better to abstract this logic into one function?

jamescrake-merani · 2026-08-12T08:50:41Z

+    return None
+
+
+def _structural_lines_from_meta(meta: dict) -> list[str]:


Some of these functions have the same issue I mentioned in metadata_summary, and perhaps could be solved in a similar way.

jamescrake-merani · 2026-08-12T08:53:07Z

+
+from PySide6 import QtWidgets
+
+from .sasbdb_api import SASBDBDatasetInfo, downloadDataset, validateDatasetId


I think it would be best to use absolute imports rather than relative.

jamescrake-merani · 2026-08-12T09:48:22Z

+        return None
+    normalized = dataset_id.strip().upper()
+    if len(normalized) != 7 or not _SASBDB_ID_PATTERN.match(normalized):
+        logger.warning("Invalid SASBDB dataset ID: %r", dataset_id)


Should this be logged as a warning considering that the user is also given a message box with the error as well?

Because at the moment, when I type in an incorrect identifier, I get both the error message, and the warning in the console:

Keep .cursor/ on disk for local AI guidance, but ignore it so it is not shipped in the PR again. Co-authored-by: Cursor <cursoragent@cursor.com>

codescene-access

Our agent can fix these. Install it.

No quality gates enabled for this code.

Quality Gate Profile: Custom Configuration
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

wpotrzebowski · 2026-08-12T14:53:31Z

@jamescrake-merani Thanks for the thorough review and for testing with multiple IDs.

Yes — this is intended for the next major release (or one after that). I’ve addressed the correctness/UX items and the smaller cleanups from your comments. Larger structural changes (dataclass/decorator display model, async download worker, moving the client out of qtgui) I’m deliberately deferring so we don’t fight the upcoming refactor.

On SASD2B2: that was a real bug. Validation was too strict (SAS + 2 letters + digits). It’s now SAS + 4 alphanumeric characters, which matches entries like SASD2B2.

jamescrake-merani

Ok thanks for your response. This all makes sense to me, and I'm happy to approve this PR in its current state.

wpotrzebowski and others added 14 commits January 14, 2026 22:00

Fetching SASBDB entries by ID and loading them into SasView

a564e61

Adding documentation

78ff44b

Updated loading metadata

7219c8f

[pre-commit.ci lite] apply automatic fixes for ruff linting errors

2b165f2

Merge branch 'main' of https://github.com/SasView/sasview into sasbdb…

177b8d6

…_load

Removing icon from file menu to make it consistent with other items

90b5f0f

Cleaned up and separated code from GuiManager

3230efc

[pre-commit.ci lite] apply automatic fixes for ruff linting errors

2d13e53

Fixing ruff issues

8556ec7

Merge branch 'sasbdb_load' of https://github.com/SasView/sasview into…

543fb41

… sasbdb_load

Fixing bugs after automatic code review

1b0f127

Moving out SASBDB-related parts out of GuiUtils

6c0573c

Updated documentation

2415149

Further separation and code clean up

e100ebc

wpotrzebowski marked this pull request as ready for review July 15, 2026 08:01

jamescrake-merani self-requested a review August 11, 2026 14:00

jamescrake-merani requested changes Aug 12, 2026

View reviewed changes

Address SASBDB PR review: fix ID validation and small cleanups

1c8b9b0

This comment was marked as outdated.

Sign in to view

Stop tracking local Cursor rules in the SASBDB branch.

6045643

Keep .cursor/ on disk for local AI guidance, but ignore it so it is not shipped in the PR again. Co-authored-by: Cursor <cursoragent@cursor.com>

codescene-access Bot approved these changes Aug 12, 2026

View reviewed changes

jamescrake-merani approved these changes Aug 12, 2026

View reviewed changes

jamescrake-merani merged commit 02130e4 into main Aug 12, 2026
37 checks passed

jamescrake-merani deleted the sasbdb_load branch August 12, 2026 15:38

Navigation Menu

Uh oh!

Sasbdb load - #3913

Sasbdb load#3913
jamescrake-merani merged 16 commits into
mainfrom
sasbdb_load

wpotrzebowski commented Apr 1, 2026 •

edited

Loading

Uh oh!

jamescrake-merani left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jamescrake-merani Aug 12, 2026

Uh oh!

Uh oh!

jamescrake-merani Aug 12, 2026

Uh oh!

jamescrake-merani Aug 12, 2026

Uh oh!

jamescrake-merani Aug 12, 2026

Uh oh!

jamescrake-merani Aug 12, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

codescene-access Bot left a comment

Uh oh!

wpotrzebowski commented Aug 12, 2026

Uh oh!

jamescrake-merani left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		return f"{rg_text} Å"


		def _format_i0_line(i0, i0_error=None, style="label") -> str:

		return None


		def _structural_lines_from_meta(meta: dict) -> list[str]:


		from PySide6 import QtWidgets

		from .sasbdb_api import SASBDBDatasetInfo, downloadDataset, validateDatasetId

Uh oh!

Conversation

wpotrzebowski commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Review Checklist

Uh oh!

jamescrake-merani left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jamescrake-merani Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jamescrake-merani Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

jamescrake-merani Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

jamescrake-merani Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

jamescrake-merani Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

codescene-access Bot left a comment

Choose a reason for hiding this comment

Uh oh!

wpotrzebowski commented Aug 12, 2026

Uh oh!

jamescrake-merani left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wpotrzebowski commented Apr 1, 2026 •

edited

Loading